Image sequences, comprising a plurality of display frames, may be transmitted between video displays for passive capture by a camera equipped device. Such transmissions may also encode data within the video image sequence. Data bits may also be presented in an image format, e.g. an animated rectangular grid pattern of black and white cells, to effect data transfer between a video display and a separate camera equipped device passively observing that display.
Transmitting data over such a display-to-device channel is complicated by the potential for data loss caused by spatial noise and/or temporal noise.
Spatial noise, caused for example by occlusion or environmental noise while each display frame of the image sequence is being captured, may result in mis-sampling of a relatively small portion of each display frame.
Temporal noise can cause one or more display frames to be incorrectly sampled or missed altogether, known as ‘dropped frames’. As a result, temporal noise results in much greater data loss or corruption since none of the data in a dropped frame is reliable.
Consider a display refreshing at D Hertz (Hz), containing encoded data animating at C Hz. The camera frame rate is R Hz. The following error cases may arise:—
Case (a): Dropped Frames.
The Nyquist limits of the display and camera are D/2 and R/2 respectively. If C>D/2, D>R/2, or C>R/2 then it is likely that the camera will fail to image a subset of frames, i.e. frames will be “dropped”.
The frequency of frame dropping is proportional to C/D, D/R and C/R but in practice C/R is the significant factor on typical equipment. Frames may also be dropped due to external factors such as misregistration of an encoded image or the environment (e.g. disruption of the line of sight between display and camera).
Case (b): Partial Refresh.
If a (raster) video display is refreshed at D Hz from top to bottom by a scanline sweep, it is possible that, at the instant when the camera samples the image, the display is only partially refreshed i.e. half the image data displayed is from the previous frame, and half is from the current (new) frame. For practical purposes, the rate at which this phenomenon occurs is proportional to the size of the image frame on the screen (typically affecting approximately 1% of frames), with insignificant correlation to D and R on typical equipment.
Case (c) Garbled Frame.
Occasionally, displayed frames will become garbled e.g. due to the camera sampling its image in the ‘idle’ (dim) phase of the display refresh cycle, or due to environmental factors. The rate of this phenomenon is typically constant and independent of D, R and C.
The problem of data loss is significant because if a receiving party fails to decode a single frame in a continually transmitted cycle of ‘n’ independent frames, a user incurs a time penalty of up to ‘n’ frames while waiting for a dropped frame to be repeated. This is disadvantageous because users desire brief interaction times when wishing to receive an image sequence.